Prediction of Pitch and Yaw Head Movements via Recurrent Neural Networks

In Virtual-Environment (VE) Applications, where virtual objects are presented in a head-mounted display, virtual images must be continuously stabilized in space against the user’s head motion. Latencies in head-motion compensation cause virtual objects to swim around instead of being stable in space. This results in an unnatural feel, disorientation, and simulation sickness in addition to errors in fitting/matching of virtual and real objects. Visual update delays are a critical technical obstacle for implementation of head-mounted displays in a wide variety of applications. To address this problem, we propose to use machine learning techniques to define a forward model of head movement based on angular velocity information. In particular, we utilize recurrent neural network to capture the temporal pattern of pitch and yaw motion. Our results demonstrate an ability to predict head motion up to 40 ms. ahead thus eliminating the main source of latencies. The accuracy of the system is tested for conditions akin to those encountered in virtual environments. These results demonstrate successful generalization by the learning system.